Investigation generation in an observation and surveillance system

ABSTRACT

The present disclosure is directed to systems and methods for generating investigations of user behavior. In an example embodiment, the system includes a video camera configured to capture video of user activity, a video analytic module to perform real-time video processing of the captured video to generate non-video data from video, and a computer configured to receive the video and the non-video data from the video camera. In some embodiments, the video camera is at least one of a traffic camera or an aerial drone camera. The computer includes a video analytics module configured to analyze one of video and non-video data to identify occurrences of particular user behavior, and an investigation generation module configured to generate an investigation containing at least one video sequence of the particular user behavior. In some embodiments, the investigation is generated in near real time. The particular user behavior may be defined as an action, an inaction, a movement, a plurality of event occurrences, a temporal event and/or an externally-generated event.

CROSS REFERENCE TO RELATED APPLICATION

The present application is a continuation-in-part of U.S. patentapplication Ser. No. 15/729,135, filed on Oct. 10, 2017, which is acontinuation of U.S. patent application Ser. No. 14/213,548, filed onMar. 14, 2014, now U.S. Pat. No. 9,786,113, which claims the benefit ofand priority to U.S. Provisional Application Ser. No. 61/798,740, filedon Mar. 15, 2013. The disclosures of each of the foregoing applicationsare hereby incorporated by reference herein in their entireties for allpurposes.

BACKGROUND 1. Technical Field

The following relates to video observation, surveillance andverification systems and methods of use. The specific application maywork in conjunction with a system providing external data such as, forexample, a point of sale (POS) transaction system that will be describedherein, however, information may be provided from any external datasystem related to transactions in health care facilities, restaurants,and the like.

2. Background of Related Art

Companies are continually trying to identify specific user behavior inorder to improve the throughput and efficiency of the company. Forexample, by understanding user behavior in the context of the retailindustry, companies can both improve product sales and reduce productshrinkage. Focusing on the latter, employee theft is one of the largestcomponents of retail inventory shrink. Therefore, companies are tryingto understand user behavior in order to reduce and ultimately eliminateinventory shrinkage.

Companies have utilized various means to prevent employee shrinkage.Passive electronic devices attached to theft-prone items in retailstores are used to trigger alarms, although customers and/or employeesmay deactivate these devices before an item leaves the store. Someretailers conduct bag and/or cart inspections for both customers andemployees while other retailers have implemented loss prevention systemsthat incorporate video monitoring of POS transactions to identifytransactions that may have been conducted in violation of implementedprocedures. Most procedures and technologies focus on identifyingindividual occurrences instead of understanding the underlying userbehaviors that occur during these events. As such, companies are unableto address the underlying condition that allows individuals to committheft.

SUMMARY

Embodiments described herein may be framed in the context of retailshrinkage, although the systems and methods described herein can beapplied to other retail or service industries such as health carefacilities, restaurants, and the like.

In one aspect, the present disclosure is directed to a system togenerate real-time investigations of user behavior. In an exampleembodiment, the system includes a video camera configured to capturevideo of user activity, a video analytic module to perform real-timevideo processing of the captured video to generate non-video data fromvideo, and a computer configured to receive the video and the non-videodata from the video camera. In some embodiments, the video camera is atleast one of a traffic camera or an aerial drone camera. The computerincludes a video analytics module configured to analyze one of video andnon-video data to identify occurrences of particular user behavior, andan investigation generation module configured to generate aninvestigation containing at least one video sequence of the particularuser behavior. In some embodiments, the investigation is generated innear real time. The particular user behavior may be defined as anaction, an inaction, a movement, a plurality of event occurrences, atemporal event and/or an externally-generated event.

In some embodiments, the investigation generation module assignsexternally-generated data to the investigation. In some embodiments, thecomputer receives the externally-generated data from a POS system andthe externally-generated data includes at least one POS transaction. Insome embodiments, the particular user behavior may be defined by a modelof the particular user behavior. In embodiments, the video analyticsmodule includes a comparator module configured to compare the model of aparticular user behavior and the non-video data.

In some embodiments, the investigation generation module is configuredto simultaneously manage and populate a plurality of investigations.

In another aspect, the present disclosure is directed to a system togenerate real-time investigations of user behavior. An exampleembodiment of the system includes a video camera configured to capturevideo of user activity, a video analytic module to perform real-timevideo processing of the captured video to generates non-video data fromvideo, and a computer configured to receive the video and the non-videodata from the video camera. In some embodiments, the video camera is atleast one of a traffic camera or an aerial drone camera. The computerincludes a video analytics module configured to analyze one of video andnon-video data to identify occurrences of particular user behavior, andan investigation generation module configured to assign a video sequencerelated to the identified occurrence of particular user behavior to aninvestigation. In some embodiments, investigation is generated in nearreal time. In some embodiments, the investigation generation moduleassigns externally-generated data to the investigation. In someembodiments, the computer receives the externally-generated data from aPOS system. The externally-generated data includes at least one POStransaction.

In some embodiments, the investigation generation module is configuredto simultaneously manage and populate a plurality of investigations. Insome embodiments, the particular user behavior is defined as at leastone of an action, an inaction, a movement, a plurality of eventoccurrences, a temporal event and an externally-generated event. In someembodiments, the particular user behavior is defined by a model of theparticular user behavior. The video analytics module includes acomparator module configured to compare the model of the particular userbehavior and the non-video data.

In yet another aspect, the present disclosure is directed to a method ofobserving behavior. In an example embodiment, the method includesreceiving video from a camera, and generating non-video data from thevideo. In some embodiments, the camera is at least one of a trafficcamera or an aerial drone camera. The non-video data includes non-videodata related to user behavior. The method includes identifying aparticular user behavior, identifying at least one occurrence of theparticular user behavior within the video data, and generating aninvestigation related to the particular user behavior.

In some embodiments, the method includes defining the particular userbehavior as at least one of an action, an inaction, a movement, aplurality of event occurrences, a temporal event, and anexternally-generated event.

In still another aspect, the present disclosure is directed tonon-transitory computer-readable medium comprising software formonitoring a point of sale (POS) transaction, which software, whenexecuted by a computer system, causes the computer system to receivevideo from a camera, generate non-video data from the video identifyinga particular user behavior, identify an occurrence of the particularuser behavior contained within the non-video data, and generate aninvestigation related to the identified occurrence of the particularuser behavior. In some embodiments, the camera is at least one of atraffic camera or an aerial drone camera. In some embodiments, theinvestigation includes video of the occurrence of the particular userbehavior contained within the non-video data. In some embodiments, thesoftware causes the computer system to receive externally-generated POStransaction data from a POS systems that includes at a least oneindividual transaction. In some embodiments, the software causes thecomputer to identify the particular user behavior temporally related tothe at least one individual transaction. In some embodiments, thesoftware causes the computer to provide data related to the at least oneindividual transaction to the investigation. In some embodiments, thesoftware causes the computer to define the particular user behavior asat least one of an action, an inaction, a movement, a plurality of eventoccurrences, a temporal event and an externally-generated event.

In a further aspect of the present disclosure, an example embodiment ofa non-transitory computer-readable medium includes software formonitoring user behavior, which software, when executed by a computersystem, causes the computer system to receive non-video data from acamera wherein the non-video data includes user behavioral informationdata, identify a particular user behavior, identify an occurrence of theparticular user behavior within the non-video data, identify video ofthe identified occurance of the particular user behavior, and generatean investigation related to the identified occurrence of the particularuser behavior, the investigation including the identified video. In someembodiments, the camera is at least one of a traffic camera or an aerialdrone camera. In some embodiments, the software causes the computer todefine the particular user behavior as at least one of an action, aninaction, a movement, a plurality of event occurrences and a temporalevent and an externally-generated event. In some embodiments, theparticular user behavior is defined by a model of the particular userbehavior. The occurrence identification step includes comparing themodel of the particular user behavior to the non-video data. In someembodiments, the software causes the computer to receiveexternally-generated data from a POS system, wherein theexternally-generated data includes at least one POS transaction, andidentify an occurrence of the particular user behavior within thenon-video data related to the at least one POS transaction. In someembodiments, the software causes the computer to provide data related tothe at least one POS in the investigation.

A “video camera” may refer to an apparatus for visual recording.Examples of a video camera may include one or more of the following: avideo imager and lens apparatus; a video camera; a digital video camera;a color camera; a monochrome camera; a camera; a camcorder; a trafficcamera, an aerial drone camera, a PC camera; a webcam; an infrared (IR)video camera; a low-light video camera; a thermal video camera; aclosed-circuit television (CCTV) camera; a pan/tilt/zoom (PTZ) camera;and a video sensing device. A video camera may be positioned to performobservation of an area of interest.

“Video” may refer to the motion pictures obtained from a video camerarepresented in analog and/or digital form. Examples of video mayinclude: television; a movie; an image sequence from a video camera orother observer; an image sequence from a live feed; a computer-generatedimage sequence; an image sequence from a computer graphics engine; animage sequence from a storage device, such as a computer-readablemedium, a digital video disk (DVD), or a high-definition disk (HDD); animage sequence from an IEEE 1394-based interface; an image sequence froma video digitizer; or an image sequence from a network.

“Video data” is a visual portion of the video.

“Non-video data” is non visual information extracted from the videodata.

A “video sequence” may refer to a selected portion of the video dataand/or the non-video data.

“Video processing” may refer to any manipulation and/or analysis ofvideo data, including, for example, compression, editing, and performingan algorithm that generates non-video data from the video.

A “frame” may refer to a particular image or other discrete unit withinvideo.

A “computer” may refer to one or more apparatus and/or one or moresystems that are capable of accepting a structured input, processing thestructured input according to prescribed rules, and producing results ofthe processing as output. Examples of a computer may include: acomputer; a stationary and/or portable computer; a computer having asingle processor, multiple processors, or multi-core processors, whichmay operate in parallel and/or not in parallel; a general purposecomputer; a supercomputer; a mainframe; a super mini-computer; amini-computer; a workstation; a micro-computer; a server; a client; aninteractive television; a web appliance; a telecommunications devicewith internet access; a hybrid combination of a computer and aninteractive television; a portable computer; a tablet personal computer(PC); a personal digital assistant (PDA); a portable telephone;application-specific hardware to emulate a computer and/or software,such as, for example, a digital signal processor (DSP), afield-programmable gate array (FPGA), an application specific integratedcircuit (ASIC), an application specific instruction-set processor(ASIP), a chip, chips, or a chip set; a system on a chip (SoC), or amultiprocessor system-on-chip (MPSoC); an optical computer; a quantumcomputer; a biological computer; and an apparatus that may accept data,may process data in accordance with one or more stored softwareprograms, may generate results, and typically may include input, output,storage, arithmetic, logic, and control units.

“Software” may refer to prescribed rules to operate a computer. Examplesof software may include: software; code segments; instructions; applets;pre-compiled code; compiled code; interpreted code; computer programs;and programmed logic.

A “computer-readable medium” may refer to any storage device used forstoring data accessible by a computer. Examples of a computer-readablemedium may include: a magnetic hard disk; a floppy disk; an opticaldisk, such as a CD-ROM and a DVD; a magnetic tape; a flash removablememory; a memory chip; and/or other types of media that may storemachine-readable instructions thereon.

A “computer system” may refer to a system having one or more computers,where each computer may include a computer-readable medium embodyingsoftware to operate the computer. Examples of a computer system mayinclude: a distributed computer system for processing information viacomputer systems linked by a network; two or more computer systemsconnected together via a network for transmitting and/or receivinginformation between the computer systems; and one or more apparatusesand/or one or more systems that may accept data, may process data inaccordance with one or more stored software programs, may generateresults, and typically may include input, output, storage, arithmetic,logic, and control units.

A “network” may refer to a number of computers and associated devicesthat may be connected by communication facilities. A network may involvepermanent connections such as cables or temporary connections such asthose made through telephone or other communication links. A network mayfurther include hard-wired connections (e.g., coaxial cable, twistedpair, optical fiber, waveguides, etc.) and/or wireless connections(e.g., radio frequency waveforms, free-space optical waveforms, acousticwaveforms, etc.). Examples of a network may include: an internet, suchas the Internet; an intranet; a local area network (LAN); a wide areanetwork (WAN); and a combination of networks, such as an internet and anintranet. Exemplary networks may operate with any of a number ofprotocols, such as Internet protocol (IP), asynchronous transfer mode(ATM), and/or synchronous optical network (SONET), user datagramprotocol (UDP), IEEE 802.x, etc.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a system block diagram of an embodiment of a videoobservation, surveillance and verification system in accordance with thepresent disclosure;

FIG. 2 is a screen-shot of an embodiment of an investigation module 200displaying an investigation in accordance with the present disclosure;

FIG. 3 is a flowchart illustrating an exemplary procedure for locatingand/or tracking a location of one or more subjects in accordance withthe present disclosure;

FIG. 4A is a perspective view of an aerial drone according to thepresent disclosure; and

FIG. 4B is a perspective view of a traffic camera according to thepresent disclosure.

DETAILED DESCRIPTION

Particular embodiments of the present disclosure are describedhereinbelow with reference to the accompanying drawings; however, it isto be understood that the disclosed embodiments are merely examples ofthe disclosure, which may be embodied in various forms. Well-knownfunctions or constructions are not described in detail to avoidobscuring the present disclosure in unnecessary detail. Therefore,specific structural and functional details disclosed herein are not tobe interpreted as limiting, but merely as a basis for the claims and asa representative basis for teaching one skilled in the art to variouslyemploy the present disclosure in virtually any appropriately detailedstructure. In this description, as well as in the drawings,like-referenced numbers represent elements which may perform the same,similar, or equivalent functions.

Additionally, the present disclosure may be described herein in terms offunctional block components, code listings, optional selections, pagedisplays, and various processing steps. It should be appreciated thatsuch functional blocks may be realized by any number of hardware and/orsoftware components configured to perform the specified functions. Forexample, embodiments of the present disclosure may employ variousintegrated circuit components, e.g., memory elements, processingelements, logic elements, look-up tables, and the like, which may carryout a variety of functions under the control of one or moremicroprocessors or other control devices.

Similarly, the software elements of the present disclosure may beimplemented with any programming or scripting language such as C, C++, C#, Java, COBOL, assembler, PERL, Python, PHP, or the like, with thevarious algorithms being implemented with any combination of datastructures, objects, processes, routines or other programming elements.The object code created may be executed on a variety of operatingsystems including, without limitation, Windows®, Macintosh OSX®, iOS®,linux, and/or Android®.

Further, it should be noted that embodiments of the present disclosuremay employ any number of conventional techniques for data transmission,signaling, data processing, network control, and the like. It should beappreciated that the particular implementations shown and describedherein are illustrative of the disclosure and its best mode and are notintended to otherwise limit the scope of the present disclosure in anyway. Examples are presented herein which may include sample data items(e.g., names, dates, etc.) which are intended as examples and are not tobe construed as limiting. Indeed, for the sake of brevity, conventionaldata networking, application development and other functional aspects ofthe systems (and components of the individual operating components ofthe systems) may not be described in detail herein. Furthermore, theconnecting lines shown in the various figures contained herein areintended to represent example functional relationships and/or physicalor virtual couplings between the various elements. It should be notedthat many alternative or additional functional relationships or physicalor virtual connections may be present in a practical electronic datacommunications system.

As will be appreciated by one of ordinary skill in the art, the presentdisclosure may be embodied as a method, a data processing system, adevice for data processing, and/or a computer program product.Accordingly, the present disclosure may take the form of an entirelysoftware embodiment, an entirely hardware embodiment, or an embodimentcombining aspects of both software and hardware. Furthermore,embodiments of the present disclosure may take the form of a computerprogram product on a computer-readable storage medium havingcomputer-readable program code means embodied in the storage medium. Anysuitable computer-readable storage medium may be utilized, includinghard disks, CD-ROM, DVD-ROM, optical storage devices, magnetic storagedevices, semiconductor storage devices (e.g., USB thumb drives) and/orthe like.

In the discussion contained herein, the terms “user interface element”and/or “button” are understood to be non-limiting, and include otheruser interface elements such as, without limitation, a hyperlink,clickable image, and the like.

Embodiments of the present disclosure are described below with referenceto block diagrams and flowchart illustrations of methods, apparatus(e.g., systems), and computer program products according to variousaspects of the disclosure. It will be understood that each functionalblock of the block diagrams and the flowchart illustrations, andcombinations of functional blocks in the block diagrams and flowchartillustrations, respectively, can be implemented by computer programinstructions. These computer program instructions may be loaded onto ageneral purpose computer, special purpose computer, mobile device orother programmable data processing apparatus to produce a machine, suchthat the instructions that execute on the computer or other programmabledata processing apparatus create means for implementing the functionsspecified in the flowchart block or blocks.

These computer program instructions may also be stored in acomputer-readable memory that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablememory produce an article of manufacture including instruction meansthat implement the function specified in the flowchart block or blocks.The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer-implemented process such that theinstructions that execute on the computer or other programmableapparatus provide steps for implementing the functions specified in theflowchart block or blocks.

Accordingly, functional blocks of the block diagrams and flowchartillustrations support combinations of means for performing the specifiedfunctions, combinations of steps for performing the specified functions,and program instruction means for performing the specified functions. Itwill also be understood that each functional block of the block diagramsand flowchart illustrations, and combinations of functional blocks inthe block diagrams and flowchart illustrations, can be implemented byeither special purpose hardware-based computer systems that perform thespecified functions or steps, or suitable combinations of specialpurpose hardware and computer instructions.

One skilled in the art will also appreciate that, for security reasons,any databases, systems, or components of the present disclosure mayconsist of any combination of databases or components at a singlelocation or at multiple locations, wherein each database or systemincludes any of various suitable security features, such as firewalls,access codes, encryption, de-encryption, compression, decompression,and/or the like.

The scope of the disclosure should be determined by the appended claimsand their legal equivalents, rather than by the examples given herein.For example, the steps recited in any method claims may be executed inany order and are not limited to the order presented in the claims.Moreover, no element is essential to the practice of the disclosureunless specifically described herein as “critical” or “essential.”

With reference to FIG. 1, a video observation, surveillance andverification system according to an embodiment of this disclosure isshown as 100. System 100 is a network video recorder that includes theability to record video from one or more cameras 110 (e.g. analog and/orIP camera) and other data obtained by way of one or more antennae 155.System 100 includes one or more video cameras 110 that connect to acomputer 120 across a connection 130. Connection 130 may be an analogconnection that provides video to the computer 120, a digital connectionthat provides a network connection between the video camera 110 and thecomputer 120, or the connection 130 may include an analog connection anda digital connection.

System 100 may include one or more video cameras 110 wherein each videocamera 110 connects to the computer 120 and a user interface 122 toprovide a user connection to the computer 120. The one or more videocameras 110 may each connect via individual connections, may connectthrough a common network connection, or through any combination thereof.

The one or more antennae 155 may be affixed to, or included within, theone or more video cameras 110 or the computer 120, and/or may be locatedremote from the one or more video cameras 110 and the computer 120. Theone or more antennae 155 may be communicatively coupled to the computer120 by way of the connection 130 or may wirelessly communicate with thecomputer 120 by way of an antenna of the computer 120.

The one or more antennae 155 may be any one or a combination of varioustypes of antennae. Example types of the one or more antennae 155 includea WiFi antenna, a media access control (MAC) antenna, a Bluetoothantenna, a cellular antenna, a near field communication antenna, a radiofrequency identification (RFID) antenna, and a global positioning system(GPS) antenna. It should be understood that the example arrangement ofthe antennae 155 shown in FIG. 1 is provided for illustrative purposesonly, and other configurations of the antennae 155 are contemplated. Forinstance, a single camera 110 may include a plurality of antennae ofdifferent types.

As discussed in more detail herein, the one or more antennae 155 areconfigured to capture mobile communication device data from one or moremobile communication devices (e.g., smartphones) located within a rangeof the one or more antennae 155 and transmit the captured mobilecommunication device data to a video data analytics module 140 forprocessing in accordance with various example embodiments herein. Theantenna 155 may be configured to capture the mobile communication devicedata by wirelessly receiving data transmitted by a mobile communicationdevice that is located within a range of the antenna. The antenna 155may be configured to wirelessly receive data from nearby mobilecommunication devices by periodically or continually pinging mobilecommunication devices and/or by being configured to periodically orcontinually listen for and capture data transmitted by nearby mobilecommunication devices without using pinging.

System 100 includes at least one video analytics module 140. A videoanalytics module 140 may reside in the computer 120 and/or one or moreof the video cameras 110. Video analytics module 140 performs videoprocessing of the video. In particular, video analytics module 140performs one or more algorithms to generate non-video data from video.Non-video data includes non-video frame data that describes content ofindividual frames such as, for example, objects identified in a frame,one or more properties of objects identified in a frame and one or moreproperties related to a pre-defined portions of a frame. Non-video datamay also include non-video temporal data that describes temporal contentbetween two or more frames. Non-video temporal data may be generatedfrom video and/or the non-video frame data. Non-video temporal dataincludes temporal data such as a temporal properties of an objectidentified in two or more frame and a temporal property of one or morepre-defined portions of two or more frames. Non-video frame data mayinclude a count of objects identified (e.g., objects may include peopleand/or any portion thereof, inanimate objects, animals, vehicles or auser defined and/or developed object) and one or more object properties(e.g., position of an object, position of any portion of an object,dimensional properties of an object, dimensional properties of portionsand/or identified features of an object) and relationship properties(e.g., a first object position with respect to a second object), or anyother object that may be identified in a frame. Objects may beidentified as objects that appear in video or objects that have beenremoved from video.

Video analytics module 140 positioned in a camera 110 converts video tovideo data and non-video data and the camera 110 and provides the videodata and the non-video data to the computer 120 over a network. As such,the system 100 distributes the video processing to the edge of thenetwork thereby minimizing the amount of processing required to beperformed by the computer 120.

Computer 120 may connect to an external system 150 that providesinformation related to the video. For example, external system 150 mayinclude a POS system that provides POS transaction information to thecomputer. Computer 120 may include a POS module 190 that receives POStransaction information and converts the POS transaction informationinto events. For example, the POS module 190 may receive datadescriptions of the content of the POS data. The POS module 190generates events based on user defined behavior discussed hereinbelow.

Computer 120 includes computer-readable medium comprising software formonitoring user behavior, which software, when executed by a computer120, causes the computer 120 to perform operations. User interface 122provides an interface to the computer 120. User interface 122 mayconnect directly to the computer 120 or connect indirectly to thecomputer 120 through a user network.

A user behavior is defined by an action, an inaction, a movement, aplurality of event occurrences, a temporal event, anexternally-generated event or any combination thereof. A particular userbehavior is defined and provided to the computer 120.

An action may include reaching for an object such as selecting a productfrom a retail shelf or retrieving an order ticket at a deli counter. Anaction may include picking up an object wherein the object has beenplaced or left at a particular location. An action may include moving aparticular object such as the opening of a door, drawer or compartment.An action may include positioning (or repositioning) a body part such asplacing a hand in a pocket after conducting retail transaction. Theaction may include moving to a particular position, a first individualengaging a second individual and/or moving a hand, arm, leg and/or footin a particular motion. An action may also include positioning a head ina particular direction, such as, for example, looking directly at amanager's office or security camera 110. Other examples are discussedhereinbelow. Actions may also include motions that result in unsanitaryconditions such as deli employees touching their faces during theprocess of filling deli orders.

Inaction may include failing to reach for an object wherein an object isdropped or positioned and the individual (e.g., object) does notretrieve the dropped object. Inaction may also include failing toperform a task that requires action, such as, for example, failing tooffer a sales receipt, change or cash back requested by a customerduring a debit transaction. Inaction may also include failing to placecash received from a customer into a register and/or cash slot. Inactionmay also include failing to make an observation that requires movementof the body or head, such as, for example, looking under a shopping cartduring a retail transaction. Inaction may also include failing to walkto a particular location or failure to perform a particular task. Forexample, confirming that a security door is locked would require theaction of approaching the door and the action of striking the door toensure that it would not open. As such, the user behavior may be definedas the inaction of approaching the door and/or the inaction of strikingthe door to confirm that the door will not open. In a health facility,an example of an inaction is failing to swipe a membership access card,indicative of a non-member entering the facility. An example of aninaction is a security guard failing to patrol assigned areas atspecified intervals or at specified times.

A movement may include movement generated by an electronic system, suchas, for example, advancing a portion of a customer receipt after aproduct is scanned or unwinding of a lottery ticket roll. Movement mayalso include the movement of items on a conveyor after a POStransaction.

A plurality of event occurrences may be a combination of relatedindividual events. For example, a plurality of events may include eventsthat require manager review or a plurality of events may include theremoval of an object from a security case and a subsequent POStransaction or a POS transaction that does not include an item locatedin the security case.

A temporal event may include the identification of a customer thatabruptly leaves a store, an individual dwelling at a store entrance orexit, an individual remaining in a particular location for an timeperiod exceeding a threshold.

Externally-generated events may include transactions provided from a POSsystem, an environmental control system (e.g., heating, cooling and/orlighting) or any other system capable of generating and/or providingexternal events. Externally-generated events may be provided via anysuitable connection, such as, for example, a hardwired connection and/ora network.

A user identifies a particular user behavior and provides and/or definescharacteristics of the particular user behavior in the computer 120.Computer 120 receives non-video data from the camera 110 wherein thenon-video data includes behavioral information data. The particular userbehavior may be defined by a model of the behavior where the modelincludes one or more attribute such a size, shape, length, width, aspectratio or any other suitable identifying or identifiable attribute. Thecomputer 120 includes a matching algorithm or module 195, such as acomparator, that compares the defined characteristics and/or model ofthe particular user behavior with user behavior in the defined n thenon-video data. Indication of a match by the matching algorithm 195generates an investigation wherein the investigation includes the videodata and non-video data identified by the matching algorithm 195.Matching algorithm 195 may be configured as an independent module orincorporated into the video analytics module 140 in the computer 120 orin any cameras 110.

A particular user behavior may be defined as the placement of acashier's hand into a pocket within a preselected period after a POStransaction. This particular user behavior is indicative of a cashierfailing to return the correct change to a customer and “pocketing” thedifference. The video analytics module 140 performs an algorithm togenerate non-video data that identifies the cashier, identifies thecashier's hand and the movement of the cashier's hand. The POS module190 using data provided from the external system 150 (e.g., POS system)identifies an event that corresponds to the completion of a POStransaction and the matching algorithm 195 searches the non-video datawithin the predetermined period of time after the completion of the POStransaction to determine if the cashier's hand is placed in theirpocket. A temporal match in a POS transaction and hand placement in apocket results in the generation of an investigation.

Video analytics module 140 may include a comparator module 192configured to compare the model of the particular user behavior and thenon-video data.

A particular user behavior may be defined as positioning a head towardan observation camera 110 exceeds a preset period or positioning of ahead directly toward a manager's office exceeds a preset period. Thisparticular user behavior is indicative of a customer trying to identifythe cameras 110 in a store in an effort to prevent being detected duringa theft or an employee trying to determine if a manager is observingtheir behavior. The video analytics module 140 performs an algorithm togenerate non-video data that identifies the head position of objects.The video analytic module 140 may also provide a vector indicating thefacial direction. The matching algorithm 195 searches the non-video datato determine if the head position and/or vector indicating facialdirection exceeds the preset period. A match results in the generationof an investigation.

A particular user behavior may be defined as a cashier failing toprovide a customer with cash back after a cash back debit card purchase(e.g., an inaction). A cash back debit transaction requires a cashier toperform two motions. The first motion is removing cash from the cashdrawer and the second motion is providing the cash to the customer.Failing to complete the first and second motions after a cash back debitcard transaction is indicative of a customer not receiving cash backfrom the transaction. The video analytics module 140 performs analgorithm to generate non-video data that identifies the cashier,identifies the cashier's hand and the movement of the cashier's hand.The POS module 190 identifies an event that corresponds to thecompletion of a cash back POS transaction and the matching algorithm 195searches the non-video data within the predetermined period after thecompletion of the POS transaction to determine if the cashier's handperformed the first and second motions. A match results in thegeneration of an investigation.

Investigations are a collection of data related to an identified event.The investigation simply documents behaviors of interest. As such,investigations require further review and investigation to understandthe particular behavior. Investigations may document customerpreferences such as why a customer selected a particular item, how acustomer shops for a particular item, and the amount of packaging detaila customer seeks before completing a selection. Other non-retailexamples include how customers select a table in a restaurant, theamount of time a customer spends reading a particular advertisement orwhich movie poster attracts customers' interests while walking to amovie.

In some instances, investigations uncover criminal activity. Forexample, an investigation generated after identifying the user behaviorof placing a cashier's hand into a pocket within a preselected periodafter a POS transaction includes a video sequence of the POStransaction. The investigation may also include a report of the POStransaction. A loss prevention individual is notified of the newlyopened investigation and the investigation can be reviewed through anysuitable user interface (e.g., computer, tablet PC, IPad, hand-heldsmart device or any other suitable device).

The loss prevention individual receives the investigation within secondsof the actual event. The video processing, POS transaction andprocessing, the matching algorithm 195 and generation of theinvestigation occur in near real time. As such, the investigation thatincludes all data and video required to view and assess the userbehavior is electronically transmitted to the loss preventionindividual's user interface.

An investigation generated after identifying the positioning of a headtoward an observation camera 110 for a preset period or positioning of ahead directly toward a manager's office for a preset period may includea plurality of video sequences related to the particular behavior. Forexample, an investigation is opened upon identification of the firstoccurrence of the user behavior. The investigation remains open andsubsequent occurrences of behavior (e.g., the same and/or other relatedbehavior) enters additional video sequences into the investigation. Theloss prevention individual is notified of the open investigation and mayobserve the further investigation while the system 100 continues topopulate the investigation with video sequences.

An investigation is populated with one or more video sequences of aparticular individual that demonstrates a user behavior or theinvestigation is populated with video sequences of any individual thatdemonstrates the user behavior. For example, shrinkage may be a resultof a single criminal or from a group of criminals working together. Assuch, the system 100 may be configured to populate the investigationwith video from any individuals that exhibit similar behavior, orinvestigations may be opened for a group of individuals that enter thestore together or make contact with each other in the store.

The system 100 provides the loss prevention individuals with tools,discussed hereinbelow, to amend and/or add to the contents of aninvestigation. For example, a loss prevention individual may add a videosequence or clip that clearly identifies the user's face, a videosequence that shows the individual entering the store and/or a videosequence that identifies additional criminal conduct. The lossprevention individual may also remove or amend a video sequenceautomatically entered into the investigation by the system 100.

An investigation may be connected to a particular employee. Employee maybe identified by an identification number, such as, for example, anidentification number entered into the POS system. The investigation mayremain open thereby forming an ongoing investigation that is populatedwith additional video sequences as behaviors of the particular employeeare observed. For example, an employee may be the target of an ongoinginternal investigation. As such, video sequences identified by thesystem are entered into the ongoing internal investigation related tothis particular employee wherein POS data is used to identify theemployee.

FIG. 2 is a screen-shot of the investigation module 200 displaying aninvestigation generated in accordance of an embodiment of thisdisclosure. Investigation module 200 is configured to generate and storeinformation required to document a particular user behavior.

Investigation module 200 includes a viewing window 210 with upper andlower viewing control bars 212 a, 212 b, a text entry window 214, atimeline window 220, a camera window 230, a search window 240, aplayback option window 250, a clip option window 260 and a filemaintenance window 270.

Investigations automatically generated by the system 100 are populatedwith information related to the particular user behavior as discussedhereinabove. For example, the investigation illustrated in FIG. 2includes a first video sequence 220 a and a second video sequence 220 bwherein the first video sequence 220 a is from the downstairs camera andthe second video sequence 220 b is from a camera located at theelevator. In one embodiment, the first video sequence 220 a was providedthrough an automatically generated investigation and the automaticallygenerated investigation was provided to the loss prevention individual.

The first video sequence 220 a is selected in the timeline window 220and played in the viewing window 210. To further this explanation andfor example, suppose the loss prevention individual, upon viewing thefirst video sequence 220 a on a PDA 123, observes an individual removinga company laptop computer from the downstairs area. In generating theinvestigation, the system identified this user behavior as a particularuser behavior and upon review, the loss prevention individual concursthat the automatically generated investigation has merit and escalatedthe automatically generated investigation to a theft investigation.

Keep in mind, the automatically generated investigation was provided tothe loss prevention individual in near real-time, therefore, theindividual now in possession of the company laptop may have only taken afew steps from where the laptop was removed.

Using the PDA 123, the loss prevention individual furthers theautomatically generated investigation (now a theft investigation) byobserving temporally related video and video data available through theinvestigation module 200 on a PDA 123.

The search window 240 may automatically select a timeframe related tothe investigation. The timeline may be manually controlled through thePDA 123.

Video and/or video data from one or more cameras listed in the camerawindow 230 may be selected for viewing in the viewing window 210. Aplurality of video streams from individual cameras (see FIG. 1) may beviewed simultaneously by selecting an alternative viewing screen fromthe upper viewing control bar 212 a.

The lower viewing control bar 212 b allows viewing video in the viewingwindow 210 in real time or other selected speeds. The investigationmodule 200 provides an investigation playback speed wherein the playbackspeed is automatically calculated to replay video at a playback speedthat requires the loss prevention individual to view every frame of thevideo sequence. Video is recorded and saved at speeds that exceed theability of a human eye to detect slight movements. Additionally, theplayback device may also have hardware and/or software limitations thatprevent the playback device from displaying every frame of video. Assuch, playback of video at “real time” results in missing individualframes of video due to human viewing limitations and/or computer displaylimitations. The investigation playback speed is calculated based on thehuman viewing limitations and the display limitations of the particulardevice being used to view the investigation module 200.

Playback option window 250 allows the video sequence and/or the videofrom each camera to be played in various modes. The all frame displaymode plays video at the calculated investigation playback speed whereinall frames are displayed and viewable during playback. The motion onlydisplay mode provides video sequences of the video that include motion.The trigger only display mode includes video sequences temporallyrelated to a trigger.

Triggers include internal triggers and/or external triggers. Internaltriggers include motion triggers defined by a user and determined by thevideo analytics module 140, POS triggers generated by the POS module 190and analytics events defined by a tripline and/or a zone (e.g., enteringand/or exiting a zone) and determined by the video analytics module 140.External triggers are generated by external hardware devices connecteddirectly or indirectly to the computer 120.

At any point of the investigation the loss prevention individual mayassign a video sequence to the timeline. For example, in FIG. 2 the lossprevention individual has added the second video sequence 220 b to theinvestigation. The second video sequence 220 b includes video providedfrom a camera positioned at the elevator and stairway. To further thescenario described hereinabove, suppose the loss prevention individualidentified a suspect carrying the laptop and approaching an elevatordisplayed in the second video sequence 220 b. In furtherance of thetheft investigation, the loss prevention individual included the secondvideo sequence 220 b in the timeline of the investigation.

Loss prevention individual may select various options from the videoclip window 260. The timeline window 220 may be populated with videoclips including one or more video sequences, a still image generatedfrom the video or text entered through the text entry window 214. Avideo clip may include a continuous video sequence. Alternatively, avideo clip using the playback option of motion only (selected in theplayback option window 250) includes a plurality of video sequences thatinclude motion (e.g., non-motion portions of the video are excluded fromthe video clip). Finally, the loss prevention individual may capture astill image of a frame to capture an individual feature such as a facialimage, a particular tool or object used during the theft, or any othersignificant image that may be required to further the investigation.

Finally, since the investigation is generated in near real-time, theloss prevention individual, upon confirmation of a theft currently inprogress, is able to notify security and apprehend the thief before theyare able to leave the premises.

In various embodiments, one or more of the video cameras 110 describedabove with reference to FIGS. 1 and 2 is an aerial drone camera 110 maybe disposed, included as a part of, or is coupled to, one or more aerialdrones 50 as shown in FIG. 4A (also sometimes referred to as unmannedaerial vehicles (UAV)). In further embodiments, the camera 110 may be atraffic camera 52 as shown in FIG. 4B, that is configured to captureimages of one or more areas and/or subjects to be tracked. The aerialdrone camera(s) 50 and/or traffic camera(s) 52 can be employed toperform various functions, such as, for example, the various functionsof the video cameras 11015 described above with reference to FIGS. 1 and2.

In another embodiment, with reference to FIG. 3, one or more aerialdrone cameras 50 and/or traffic cameras 52 may be employed, inconjunction with one or more other sources of information in someinstances, to perform a method 300 for locating and/or tracking alocation of one or more subjects, such as a person who has been detectedas having committed a crime at a particular location, across regionsthat correspond to one or more networks, such as an aerial drone cameranetwork, a traffic camera network, a store camera network, and/or othertypes of networks. In this manner, communication among multiple nodesand/or networks, including nodes and/or networks that employ aerialdrone cameras and/or traffic cameras, can cooperate to facilitate moreeffective location of subjects and/or tracking of locations of subjects.

At 302, a behavior of a subject is detected in a region, such as aretail store premises, that corresponds to a first network, such as anetwork including cameras 110, antennas 155, and/or the like. Althoughthe method 300 is described in the context of a single subject orperson, the method 300 is also applicable to multiple subjects, such asa group of people who are acting together or separately. Exemplary typesof behaviors that can be detected at 302 include, without limitation, anaction, an inaction, a movement, a plurality of event occurrences, atemporal event, an externally-generated event, the commission of atheft, the leaving of an unattended package, the commission of violence,the commission of a crime, and/or another type of behavior. In someexample embodiments, in addition to, or as an alternative to, detectinga behavior of a subject at 302, an abnormal situation is detected, suchas an abnormal condition (pre programmed condition(s)), an abnormalscenario (loitering, convergence, separation of clothing articles orbackpacks, briefcases, groceries for abnormal time, etc.) or otherscenarios based on behavior of elements (customers, patrons, people incrowd, etc.) in one or multiple video streams. For the sake ofillustration, the description of the method 300 is provided in thecontext of detecting a behavior of a subject at 302, but the method 300is similarly applicable to detecting an abnormal situation at 302.

Detection of the behavior of the subject includes obtaining informationfrom one or more source(s), such as video and/or image information ofthe subject obtained via one or more video cameras 110 installed at ornear a premises, non-video information (e.g., mobile communicationdevice data) obtained from one or more antennas 155 installed at or nearthe premises, information provided by an employee or witness by way of acomputer 120 at the premises, and/or other types of information obtainedfrom other types of sources at or near the premises. Based on theobtained information, the behavior can be detected by way of the cameras110 (in the case of smart cameras with such processing capability),and/or by a computer 120 or a server that is communicatively coupled tothe cameras 110.

In various embodiments, there may be multiple types of cameras 110, suchas smart cameras 110 that have processing capabilities to perform one ormore of the functions described in connection with the method 300, andnon-smart cameras that lack processing capabilities to perform one ormore of the functions described in connection with the method 300. Ingeneral, any one or more of the functions described in connection withthe method 300 may be performed in a centralized manner by one or moreof the cameras (or other components of networks), and/or in adistributed manner by one or more of the cameras 110 and/or the computer120, and/or the like. Additionally, the cameras, computers, and/or othercomponents are configured, in some aspects, to communicate with oneanother to cooperate to execute the various functions of the method 400.For instance, in the event that a non-smart camera lacks processingcapabilities to perform one or more of the functions described inconnection with the method 400 (for example, a particular matchingalgorithm), the non-smart camera may communicate information (such as,for example, raw video data) to a smart camera and/or to a computer orother device that has the processing capabilities to perform the one ormore particular functions described in connection with the method 400,so that the function(s) can be performed. Further, the non-smart cameramay, in some aspects, forward to the smart camera, computer, or otherdevice, information enabling the non-smart camera to be identified, sothat if the non-smart camera captures an image of the subject, thelocation of the non-smart camera can be traced back and a location ofthe subject can be ascertained.

At 304, one or more attributes of the subject, or associated with thesubject, are obtained from one or more sources. For example, anattribute of a face of the subject may be obtained by way of an imagecaptured by way of a video camera 110, an attribute (e.g., a color, atype, and/or the like) of a clothing item that the subject is wearingcan be obtained by way of an image captured by way of a video camera110, mobile communication device data and/or a wireless signature of amobile communication device or PDA 123 that the subject is carrying canbe obtained by way of an antenna 155, and/or the like.

At 306, the one or more attributes that are associated with the subjectand were obtained at 304 are transmitted or pushed to one or more othernodes (e.g., video cameras 110, antennas 155, and/or other devicesresident on one or more networks) and/or networks, for instance, toenable those other nodes and/or networks to locate the subject and/ortrack a location of the subject. The attribute(s) can be transmitted toone or more nodes and/or networks by way of the network, or any suitablewired and/or wireless communication path or network.

At 308, a tracking loop is initiated to track a location of the subjectwithin a first region that corresponds to the first network. Thetracking loop, in some embodiments, includes performing the proceduresdescribed below in connection with 310, 312, 314, 316, 318, 320, and 322for the particular region in which the tracking is commencing. In oneexample, the first region is the region where the behavior of thesubject was initially detected at 302. For instance, the first regionmay be a retail store premises and the first network may be a network ofthe video cameras 110, the antennas 155, and/or the like that areinstalled at or near the first region. In some example embodiments, thetracking loop is performed in parallel for multiple regions (e.g., byemploying multiple nodes and/or networks, such as networks of aerialdrone cameras, traffic cameras, store premises, and/or the like) in tofacilitate more comprehensive tracking of the location of the subjectand/or to facilitate tracking of the location of the subject across awide area. In a further embodiment, the tracking loop is performed inparallel for multiple regions corresponding to multiple networks, andthe multiple networks collaborate in tracking the location of thesubject to share the processing load and/or provide more accurate orrapid tracking results.

At 310, updated and/or more recent data associated with the subject isaggregated from various sources, such as one or more of the cameras 110,antennas 155, and/or other sources. Example types of data that can beaggregated at 310 include, without limitation, a facial image of thesubject, an image of clothing worn by the subject, mobile communicationdevice data and/or a wireless signature of a mobile communication deviceor PDA 123 carried by the subject, and/or other types of data.

At 312, a determination is made as to whether one or more items of datathat were aggregated at 310 match the one or more attributes that wereobtained at 304. For example, the determination at 312 may includecomparing one or more items of data that were aggregated at 310 to theone or more attributes that were obtained at 304 to determine whethermore recently captured data (such as, image data, video data, wirelesscommunication data, and/or other types of data) correspond to thesubject. In this manner, the determination at 312 can indicate whetherthe location of the subject in a particular region is still successfullybeing tracked, or whether the location of the subject is no longersuccessfully being tracked in the particular region and so a widerscoped search may be needed. In one example, the determination at 312includes comparing an attribute (e.g., of a facial image) of the subjectthat was obtained at 304 to an attribute (e.g., of a facial image) of aperson whose image was captured subsequent to the obtaining of theattribute at 304 (and, in some instance, by way of a different videocamera 110) to determine whether the person whose image was subsequentlycaptured matches the subject, thereby indicating that the location ofthe subject is still successfully being tracked.

In some embodiments, multiple types of attribute categories are arrangedin hierarchical tiers according to complexity of processing required indetecting a match at 312. For example, a first tier of attributes forwhich the processing complexity required for detecting a match at 312 isminimal may include a clothing color or hair color associated with thesubject. A second tier of attributes for which the processing complexityrequired for detecting a match at 312 is greater than that of the firsttier of attributes may include mobile communication device data and/orwireless information relating to a mobile communication device carriedby the subject and/or registered to the subject. A third tier ofattributes for which the processing complexity required for detecting amatch is even greater than that of the first and second tiers ofattributes may include a gait of the subject. In this manner, dependingon the tiers of attributes being employed for the matching at 312,and/or depending on the processing capabilities of the cameras 110,nodes, and/or other sources, processing of the matching at 312 can beredirected for completion by the appropriate device.

Referring now back to 312, if it is determined at 312 that one or moreitems of data that were aggregated at 310 match the one or moreattributes that were obtained at 304 (“YES” at 312), then the method 300progresses to 314. At 314, a location of the subject is determined basedat least in part on the information aggregated at 310 and/or on otherinformation. For example, the determining of the location of the subjectat 314 includes, in some embodiments, computing a location of thesubject based on a location of the camera 110 (or other source) fromwhich the information was aggregated at 310.

At 316, information relating to the tracking of the location of thesubject is displayed to a user (for example, a police officer or otheremergency personnel) by way of a user interface, such as a graphicaluser interface (GUI). The GUI, in some examples, includes a map overwhich an overlay is displayed indicating a location of the subject beingtracked. The GUI may also include additional information, such as one ormore of the attributes of the subject being tracked, including forinstance, a facial image of the subject obtained by way of one or moreof the cameras 110, attributes of clothing worn by the user, anattribute of a mobile communication device carried by the user, a nameor other information identifying the user generated, for instance, bymatching the captured facial image of the subject to a facial imagestored in a database of facial images, and/or the like. In this manner,the GUI enables the user to continually track the location of thesubject throughout multiple regions that may correspond to multiplenodes and/or networks.

At 318, a determination is made as to whether any additional attributeassociated with the subject being tracked is available. In someexamples, the determination at 318 is based at least in part on one ormore items of information-such as images of the subject, video of thesubject, mobile communication device data and/or wireless signatures ofmobile communication devices or PDAs 123 carried by the subject, and/orthe like—that have been obtained thus far by way of the camera(s) 110,the antenna(s) 155, and/or other source(s). Example types of additionalattributes that may be available include, without limitation, additionalattributes of facial images captured of the subject having differentangles and/or providing information beyond the information of previouslyobtained and recorded attributes, an attribute, such as a make, model,color, license plate number, of a vehicle that the subject has enteredand is traveling in, and/or the like. By determining whether anyadditional attribute associated with the subject being tracked isavailable, a more comprehensive and robust profile of the subject may becompiled, thereby facilitating more accurate and efficient tracking ofthe location of the subject.

If it is determined at 318 that any additional attribute associated withthe subject being tracked is available (“YES” at 318), then the method300 proceeds to 320. At 320, the additional attribute associated withthe subject being tracked is obtained by way of the camera(s) 110, theantenna(s) 155, and/or the other source(s), and is stored in a memoryfor later use. At 322, the additional attribute that was obtained at 320is transmitted or pushed to one or more other nodes and/or networks, forinstance, to enable those other nodes and/or networks to moreeffectively locate the subject and/or track a location of the subject.From 322, or if it is determined at 318 that no additional attributeassociated with the subject being tracked is available (“NO” at 318),then the method 300 proceeds back to 310 to aggregate updated and/ormore recent data associated with the subject to continually track thelocation of the subject throughout the region.

In some embodiments, at 318, in addition or as an alternative todetermining whether any additional attribute associated with the subjectbeing tracked is available, a determination is made as to whether anyattribute associated with the subject being tracked has changed. Forexample, in some cases the subject may be tracked based on multipleattributes, such as a hair color, a clothing color, a height, a vehiclemake, a vehicle model, a vehicle color, a vehicle license plate, mobilecommunication device data, and/or the like. The multiple attributes mayoriginate from a variety of sources, such as an image of the subjectpreviously captured by the video camera(s) 110, mobile communicationdevice information previously captured by the antenna(s) 155,intelligence provided by law enforcement personnel, and/or the like. Inthis manner, when an image of a person is obtained by way of the cameras110 and/or mobile communication device information associated with aperson is obtained by way of the antennas(s) 155, the person can beidentified as matching the subject who is being tracked with a degree ofconfidence that is proportional to the number of attributes of theperson that are detected in the image as matching the multipleattributes that serve as the basis upon which the subject is beingtracked. In some cases, one of the attributes of the subject may change.For example, the subject may remove a wig, change vehicles, changeclothing, and/or the like in an effort to elude tracking and capture. Insuch cases, it may be determined at 318 that one or more of the multipleattributes have changed. In particular, if the cameras 110 and/orantennas 155 are no longer able to detect a person matching all of themultiple (for example, five) attributes being tracked, then the computer120 may search for a person matching a lesser number (for example, fouror fewer) of the attributes that were previously being tracked. If aperson matching the lesser number of the attributes is detected by oneor more of the cameras 110 and/or antennas 155, then that person may beflagged as a secondary subject to be tracked simultaneously whilesearching for the primary subject having attributes that match all themultiple attributes being tracked. If the person matching all of themultiple attributes is no longer locatable by the images captured viathe cameras 110 and/or the information obtained by the antennas 155,then the secondary subject matching the lesser number of the attributesmay be promoted to be the primary subject so that tracking resources maybe appropriately and effectively allocated. In some cases, the change inattribute is verified before the secondary subject is promoted to beingthe primary subject. For example, the change in attribute may beverified by the processing of images captured via the cameras 110, whichdetect the subject discarding a clothing item or a wig. Alternatively,the change in attribute may be verified by law enforcement personnel wholocate the discarded clothing item or wig. In this regard, the computer120 may provide a location and time information to law enforcementpersonnel based on the last known or tracked location of the primarysubject matching all of the multiple attributes, to enable the lawenforcement to dispatch personnel to the location to conduct theverification. Additionally, when the subject is being tracked acrossmultiple networks, the system 100 can push the updated list ofattributes (for example, the lesser number of attributes) to one or moreother nodes (e.g., cameras 110, antennas 155, and/or other devicesresident on one or more networks) and/or networks. This facilitatesimproved adaptive tracking of subjects across multiple networks evenwhen the subjects are expending effort to change their image to eludetracking and capture.

Referring back to 312, if it is determined that the one or more items ofdata that were aggregated at 310 do not match the one or more attributesthat were obtained at 304 (“NO” at 312), then the method 300 proceeds to324. At 324, a determination is made as to whether the subject hasdeparted the region in which the subject previously was being tracked,for instance, the region corresponding to the premises at which thebehavior was detected at 302. In some embodiments, the determination at324 is based on the amount of time that has elapsed since the locationof the subject was successfully being tracked. In particular, if theamount of time that has elapsed since the location of the subject wassuccessfully being tracked exceeds a predetermined threshold, then it isdetermined at 324 that the subject has departed the region, and if theamount of time that has elapsed since the location of the subject wassuccessfully being tracked does not exceed the predetermined threshold,then it is determined at 324 that the subject has not departed theregion.

If it is determined at 324 that the subject has not departed the regionin which the subject previously was being tracked (“NO” at 324), thenthe method 300 proceeds back to 310 to aggregate updated and/or morerecent data associated with the subject to continually track thelocation of the subject throughout the region. If, on the other hand, itis determined at 324 that the subject has departed the region in whichthe subject previously was being tracked (“YES” at 324), then the method300 progresses to 326. At 326, an alert is communicated to one or moreother nodes and/or networks, by way of one or more wired and/or wirelesscommunication paths, indicating that the subject has departed the firstregion in which the subject previously was being tracked, for instance,the region corresponding to the premises at which the behavior wasdetected at 302. In some embodiments, the alert is provided to a widearea of nodes and/or networks that are adjacent and/or proximal to theregion in which the subject previously was being tracked. In thismanner, the additional neighboring nodes and/or networks can attempt tolocate the subject and/or track a location of the subject.

In some embodiments, the alert is provided to a select set of nodesand/or networks based on one or more factors that enable more efficientallocation of tracking resources. For example, a determination may bemade as to whether any traffic cameras in the region have detected atraffic law violation, such as driving through a red light. If a trafficcamera in the region has detected a traffic law violation, then, basedon a prediction that the traffic law violation may have been committedby the subject fleeing the scene of a crime, the alert may be providedto one or more nodes and/or networks that overlap with a region of thetraffic camera in an effort to quickly locate the customer without theneed to utilize a wide array of cameras and/or other resources. Inaddition, based on the detection at 324 that the subject has departedthe region in which the subject previously was being tracked, police orother emergency personnel can launch one or more aerial drone cameras110 that can communicate attributes and other information with oneanother to facilitate a collaborative search plan, based in part on oneor more neighboring regions of interest, to identify and/or track alocation of the subject.

At 328, a determination is made as to whether the searching for, and/ortracking of, the location of the subject is concluded. In someembodiments, the determination at 328 is based on whether an instructionhas been received from a police officer or other emergency personnelindicating that the search for the subject has been concluded, forinstance, in a case where the subject has been apprehended and is inpolice custody. If it is determined at 328 that the searching for,and/or tracking of, the location of the subject is not concluded (“NO”at 328), then the method 300 proceeds to 330 where a tracking loop isinitiated to identify and/or track a location of the subject within asecond region that corresponds to a second network. The tracking loop,in some embodiments, includes performing the procedures described abovein connection with 310, 312, 314, 316, 318, 320, and 322 for theparticular region in which the tracking is commencing. If, on the otherhand, it is determined at 328 that the searching for, and/or trackingof, the location of the subject is concluded (“YES” at 328), then themethod 300 ends.

As various changes could be made in the above constructions withoutdeparting from the scope of the disclosure, it is intended that allmatter contained in the above description shall be interpreted asillustrative and not in a limiting sense. It will be seen that severalobjects of the disclosure are achieved and other advantageous resultsattained, as defined by the scope of the following claims.

What is claimed is:
 1. A system to generate real-time investigations ofuser behavior comprising: a video camera configured to capture video ofuser activity to generate video data, wherein the video camera is atleast one of a traffic camera or an aerial drone camera; and a computerconfigured to receive the video data from the video camera, the computerincluding: a video analytics module configured to: perform real-timevideo processing of the video data to generate non-video data; andanalyze one of video and non-video data to identify occurrences ofparticular user behavior; and an investigation generation moduleconfigured to generate an investigation containing at least one videosequence of the particular user behavior.
 2. The system according toclaim 1, wherein the investigation is generated in real time.
 3. Thesystem according to claim 1, wherein the investigation generation moduleis configured to assign externally-generated data to the investigation.4. The system according to claim 3, further comprising a POS system,wherein the computer receives the externally-generated data from the POSsystem and the externally-generated data includes at least one POStransaction.
 5. The system according to claim 1, wherein theinvestigation generation module is configured to simultaneously manageand populate a plurality of investigations.
 6. The system according toclaim 1, wherein the particular user behavior is defined as at least oneof an action, an inaction, a movement, a plurality of event occurrences,a temporal event and an externally-generated event.
 7. The systemaccording to claim 1, wherein the particular user behavior is defined bya model of the particular user behavior and the video analytics modulefurther includes: a comparator module configured to compare the model ofthe particular user behavior and the non-video data.
 8. A system togenerate real-time investigations of user behavior comprising: a videocamera configured to capture video of user activity to generate videodata, wherein the video camera is at least one of a traffic camera or anaerial drone camera; and a computer configured to receive the video datafrom the video data from the video camera, the computer including: avideo analytics module configured to: perform real-time video processingof the video data to generate non-video data; and analyze one of videoand non-video data to identify at least one occurrence of particularuser behavior; and an investigation generation module configured toassign a video sequence related to the at least one identifiedoccurrence of particular user behavior to an investigation.
 9. Thesystem according to claim 8, wherein the investigation is generated innear real time.
 10. The system according to claim 8, wherein theinvestigation generation module is configured to assignexternally-generated data to the investigation.
 11. The system accordingto claim 10, further comprising a POS system, wherein the computerreceives the externally-generated data from the POS system and theexternally-generated data includes at least one POS transaction.
 12. Thesystem according to claim 8, wherein the investigation generation moduleis further configured to simultaneously manage and populate a pluralityof investigations.
 13. The system according to claim 8, wherein theparticular user behavior is defined as at least one of an action, aninaction, a movement, a plurality of event occurrences, a temporal eventand an externally-generated event.
 14. The system according to claim 8,wherein the particular user behavior is defined by a model of theparticular user behavior and the computer further includes: a comparatormodule configured to compare the model of the particular user behaviorand the non-video data.
 15. A method of observing behavior comprisingthe steps of: receiving video from a camera, wherein the camera is atleast one of a traffic camera or an aerial drone camera; generatingnon-video data from the video wherein the non-video data includesnon-video data related to user behavior; identifying a particular userbehavior; identifying at least one occurrence of the particular userbehavior within the video data; and generating an investigation relatedto the particular user behavior.
 16. The method according to claim 15,wherein the identifying the particular user behavior further includes:defining the particular user behavior as at least one of an action, aninaction, a movement, a plurality of event occurrences, a temporalevent, and an externally-generated event.
 17. Non-transitorycomputer-readable medium comprising software for monitoring a point ofsale (POS) transaction, which software, when executed by a computersystem, causes the computer system to perform operations comprising amethod of: receiving video from a camera, wherein the camera is at leastone of a traffic camera or an aerial drone camera; generating non-videodata from the video; identifying a particular user behavior; identifyingan occurrence of the particular user behavior contained within thenon-video data; and generating an investigation related to theidentified occurrence of the particular user behavior.
 18. The medium asset forth in claim 17, wherein the investigation includes video of theoccurrence of the particular user behavior contained within thenon-video data.
 19. The medium as set forth in claim 17, wherein themethod further includes the steps of: receiving externally-generated POStransaction data from a POS systems wherein the POS transaction dataincludes at least one individual transaction; identifying the particularuser behavior temporally related to the at least one individualtransaction; and providing data related to the at least one individualtransaction to the investigation.
 20. The medium as set forth in claim17, wherein the method further includes the step of: defining theparticular user behavior as at least one of an action, an inaction, amovement, a plurality of event occurrences, a temporal event and anexternally-generated event.